1. Field of the Invention
This invention relates generally to the field of computer graphics and, more particularly, to high performance graphics systems.
2. Description of the Related Art
A computer system typically relies upon its graphics system for producing visual output on the computer screen or display device. Early graphics systems were only responsible for taking what the processor produced as output and displaying it on the screen. In essence, they acted as simple translators or interfaces. Modem graphics systems, however, incorporate graphics processors with a great deal of processing power. They now act more like coprocessors rather than simple translators. This change is due to the recent increase in both the complexity and amount of data being sent to the display device. For example, modem computer displays have many more pixels, greater color depth, and are able to display images that are more complex with higher refresh rates than earlier models. Similarly, the images displayed are now more complex and may involve advanced techniques such as anti-aliasing and texture mapping.
As a result, without considerable processing power in the graphics system, the CPU would spend a great deal of time performing graphics calculations. This could rob the computer system of the processing power needed for performing other tasks associated with program execution and thereby dramatically reduce overall system performance. With a powerful graphics system, however, when the CPU is instructed to draw a box on the screen, the CPU is freed from having to compute the position and color of each pixel. Instead, the CPU may send a request to the video card stating: xe2x80x9cdraw a box at these coordinates.xe2x80x9d The graphics system then draws the box, freeing the processor to perform other tasks.
Generally, a graphics system in a computer (also referred to as a graphics system) is a type of video adapter that contains its own processor to boost performance levels. These processors are specialized for computing graphical transformations, so they tend to achieve better results than the general-purpose CPU used by the computer system. In addition, they free up the computer""s CPU to execute other commands while the graphics system is handling graphics computations. The popularity of graphics applications, and especially multimedia applications, has made high performance graphics systems a common feature of computer systems. Most computer manufacturers now bundle a high performance graphics system with their systems.
Since graphics systems typically perform only a limited set of functions, they may be customized and therefore are far more efficient at graphics operations than the computer""s general-purpose central processor. While early graphics systems were limited to performing two-dimensional (2D) graphics, their functionality has increased to support three-dimensional (3D) wire-frame graphics, 3D solids, and now includes support for three-dimensional (3D) graphics with textures and special effects such as advanced shading, fogging, alpha-blending, and specular highlighting.
The processing power of 3D graphics systems has been improving at a breakneck pace. A few years ago, shaded images of simple objects could only be rendered at a few frames per second, while today""s systems support rendering of complex objects at 60 Hz or higher. At this rate of increase, in the not too distant future, graphics systems will literally be able to render more pixels than a single human""s visual system can perceive. While this extra performance may be useable in multiple-viewer environments, it may be wasted in more common primarily single-viewer environments. Thus, a graphics system is desired which is capable of matching the variable nature of the human resolution system (i.e., capable of putting the quality where it is needed or most perceivable).
While the number of pixels is an important factor in determining graphics system performance, another factor of equal import is the quality of the image. For example, an image with a high pixel density may still appear unrealistic if edges within the image are too sharp or jagged (also referred to as xe2x80x9caliasedxe2x80x9d). One well-known technique to overcome these problems is anti-aliasing. Anti-aliasing involves smoothing the edges of objects by shading pixels along the borders of graphics elements. More specifically, anti-aliasing entails removing higher frequency components from an image before they cause disturbing visual artifacts. For example, anti-aliasing may soften or smooth high contrast edges in an image by forcing certain pixels to intermediate values (e.g., around the silhouette of a bright object superimposed against a dark background).
Another visual effect used to increase the realism of computer images is alpha blending. Alpha blending is a technique that controls the transparency of an object, allowing realistic rendering of translucent surfaces such as water or glass. Another effect used to improve realism is fogging. Fogging obscures an object as it moves away from the viewer. Simple fogging is a special case of alpha blending in which the degree of alpha changes with distance so that the object appears to vanish into a haze as the object moves away from the viewer. This simple fogging may also be referred to as xe2x80x9cdepth cueingxe2x80x9d or atmospheric attenuation, i.e., lowering the contrast of an object so that it appears less prominent as it recedes. More complex types of fogging go beyond a simple linear function to provide more complex relationships between the level of translucence and an object""s distance from the viewer. Current state of the art software systems go even further by utilizing atmospheric models to provide low-lying fog with improved realism.
While the techniques listed above may dramatically improve the appearance of computer graphics images, they also have certain limitations. In particular, they may introduce their own aberrations and are typically limited by the density of pixels displayed on the display device.
As a result, a graphics system is desired which is capable of utilizing increased performance levels to increase not only the number of pixels rendered but also the quality of the image rendered. In addition, a graphics system is desired which is capable of utilizing increases in processing power to improve the results of graphics effects such as anti-aliasing.
Prior art graphics systems have generally fallen short of these goals. Prior art graphics systems use a conventional frame buffer for refreshing pixel/video data on the display. The frame buffer stores rows and columns of pixels that exactly correspond to respective row and column locations on the display. Prior art graphics system render 2D and/or 3D images or objects into the frame buffer in pixel form, and then read the pixels from the frame buffer during a screen refresh to refresh the display. Thus, the frame buffer stores the output pixels that are provided to the display. To reduce visual artifacts that may be created by refreshing the screen at the same time the frame buffer is being updated, most graphics systems"" frame buffers are double-buffered.
To obtain more realistic images, some prior art graphics systems have gone further by generating more than one sample per pixel. As used herein, the term xe2x80x9csamplexe2x80x9d refers to calculated information that indicates one or more of the color, depth (z), transparency, and potentially other information, of a particular point on an object or image. For example, a sample may comprise the following component values: a red value, a green value, a blue value, a z value, and an alpha value (e.g., representing the transparency of the sample). A sample may also comprise other information, e.g., a z-depth value, a blur value, an intensity value, brighter-than-bright information, and an indicator that the sample consists partially or completely of control information rather than color information (i.e., xe2x80x9csample control informationxe2x80x9d). By calculating more samples than pixels (i.e., super-sampling), a more detailed image is calculated than can be displayed on the display device. For example, a graphics system may calculate four samples for each pixel to be output to the display device. After the samples are calculated, they are then combined or filtered to form the pixels that are stored in the frame buffer and then conveyed to the display device. Using pixels formed in this manner may create a more realistic final image because overly abrupt changes in the image may be smoothed by the filtering process.
These prior art super-sampling systems typically generate a number of samples that are far greater than the number of pixel locations on the display. These prior art systems typically have rendering processors that calculate the samples and store them into a render buffer. Filtering hardware then reads the samples from the render buffer, filters the samples to create pixels, and then stores the pixels in a traditional frame buffer. The traditional frame buffer is typically double-buffered, with one side being used for refreshing the display device while the other side is updated by the filtering hardware. Once the samples have been filtered, the resulting pixels are stored in a traditional frame buffer that is used to refresh the display device. These systems, however, have generally suffered from limitations imposed by the conventional frame buffer and by the added latency caused by the render buffer and filtering. Therefore, an improved graphics system is desired which includes the benefits of pixel super-sampling while avoiding the drawbacks of the conventional frame buffer.
U.S. patent application Ser. No. 09/251,453 titled xe2x80x9cGraphics System with Programmable Real-Time Sample Filteringxe2x80x9d discloses a computer graphics system that utilizes a super-sampled sample buffer and a sample-to-pixel calculation unit for refreshing the display. The graphics processor generates a plurality of samples and stores them into a sample buffer. The graphics processor preferably generates and stores more than one sample for at least a subset of the pixel locations on the display. Thus, the sample buffer is a super-sampled sample buffer which stores a number of samples that may be far greater than the number of pixel locations on the display. The sample-to-pixel calculation unit is configured to read the samples from the super-sampled sample buffer and filter or convolve the samples into respective output pixels, wherein the output pixels are then provided to refresh the display. The sample-to-pixel calculation unit selects one or more samples and filters them to generate an output pixel. The sample-to-pixel calculation unit may operate to obtain samples and generate pixels that are provided directly to the display with no frame buffer therebetween.
While super-sampling may create a more realistic final image, the effects of super-sampling may not always be desired. For example, it may be desirable to display areas and/or windows of the display, such as individual windows in a graphics user interface, with one sample per pixel. This may be desirable, for example, to provide a crisper, sharper image in a window. Alternatively, the system may be required to display images and/or the contents of certain windows that have already been anti-aliased by another graphics processor. In those cases, it would be preferable to not attempt to further anti-alias the image as that would result in loss of sharpness and contrast.
In addition, super-sampling may not be compatible with existing windowing systems such as the X-Windows system. For example, an X-Windows system typically provides a plurality of pixel write commands to a receiving or client system, which indicates where pixels will be written into a traditional frame buffer. Therefore, a system with a super-sampled sample buffer may desire to emulate a traditional frame buffer in order to enable proper receipt and processing of pixel write commands received from an X-Windows system. In addition, certain users of legacy systems such as X-Windows expect the display to appear a certain way, e.g., expect the display output to be aliased. It would thus be preferable to keep the visual output of systems such as X-Windows unchanged. Therefore, in a system with a super-sampled sample buffer, the system may be required to receive pixel write commands or other graphics commands from legacy systems that assume the presence of a traditional frame buffer. In this instance, it may be necessary or desirable for the super-sampled sample buffer graphic system to emulate a traditional frame buffer in order to properly receive and execute these commands which assume a traditional prior art frame buffer.
It would further be desirable to have a method for designating a sample mode for graphics data in a super-sampled sample buffer such as on a per window or per object basis. Such a method could enable the graphics processor to define the behavior of all samples in a pixel for a given window or object. For example, the graphics processor could identify whether the graphics data being rendered is to utilize multi-sampling or only a single sample per pixel on a per window basis.
As a result, a graphics system is desired which is capable of supporting a single sample per pixel in a super-sampled environment. In addition, a graphics system is desired which is capable of efficiently defining the behavior of all samples in a pixel for a given window.
The present invention comprises a computer graphics system that utilizes a super-sampled sample buffer and one or more programmable sample-to-pixel calculation units for refreshing the display. In one embodiment, the graphics system may have a graphics processor, a super-sampled sample buffer, and a sample-to-pixel calculation unit. In some embodiments, the graphics system may have the ability to use different sample modes for different windows and/or objects on the display.
The graphics processor may generate a plurality of samples and stores the samples into a sample buffer. The graphics processor preferably generates and stores more than one sample for at least a subset of the pixel locations on the display. Thus, the sample buffer is generally a super-sampled sample buffer which stores a number of samples that, in some embodiments, may be far greater than the number of pixel locations on the display. In other embodiments, the total number of samples may be closer to, equal to, or even less than the total number of pixel locations on the display device, but the samples may be more densely positioned in certain areas and less densely positioned in other areas.
The sample-to-pixel calculation unit is configured to read the samples from the super-sampled sample buffer and filter or convolve the samples into respective output pixels, wherein the output pixels are then provided to refresh the display. The sample-to-pixel calculation unit selects one or more samples and filters them to generate an output pixel. The number of samples selected and/or filtered by the sample-to-pixel calculation unit is typically greater than one, but this may vary depending upon the exact implementation.
The sample-to-pixel calculation unit may access the samples from the super-sampled sample buffer, perform a filtering operation, and then provide the resulting output pixels directly to the display, preferably on-the-fly. The graphics system may operate without a conventional frame buffer, i.e., the graphics system may not utilize a conventional frame buffer that stores the actual pixel values that are being refreshed on the display. Thus, the sample-to-pixel calculation units may calculate each pixel for each screen refresh on a real-time basis or on an on-the-fly basis. The display may then receive and display the output pixels.
The present invention allows for different windows and/or objects of the display to be displayed using different sample modes. Note that, as used herein, the terms xe2x80x9cwindowxe2x80x9d and xe2x80x9cobjectxe2x80x9d are used interchangeably and refer to a region of the display.
In one embodiment, the sample mode for each window is defined by examining a window ID associated with each window. In one embodiment, the graphics system determines the sample mode on a per sample, per pixel, or per bin basis by examining a window ID that may be associated with each. Thus, the window ID information for each window may be associated with each sample, each bin, or each pixel depending on the desired granularity. The graphics processor may maintain a look-up table where an association is stored between each of the windows on the display and the sample mode for each window. Example sample modes include multiple samples per pixel (default mode), single sample per pixel, two samples per pixel, four samples per pixel, variable samples per pixel, etc.
In addition, the look-up table may maintain other information about each window such as filter mode, color attributes, and source attributes. Examples of color attributes may include a true color/pseudo-color attribute, a gamma-correction attribute, an anti-aliasing attribute, a depth-of-field attribute, and a brighter-than-bright attribute. Examples of source attributes may include a double-buffer attribute and an overlay attribute. Examples of filter mode may include an averaging filter mode (e.g., one requiring no multiplies), a convolution filter mode (e.g., a re-sampling band-pass filter), a summing filter mode, a weighting filter mode, and a randomized function filter mode. Thus, the window ID may specify a number of parameters or characteristics for pixels in the window, including sample mode, color attributes, source attributes, filter type, etc.
In one embodiment, the default sample mode comprises rendering multiple samples per pixel and applying a filter to selected samples to generate output pixels for the display as described above. The default mode is used for the whole display unless another sample mode is specified for a particular window. In some cases, it may be desirable to use alternate sample modes such as a xe2x80x9csingle sample per pixelxe2x80x9d mode for certain windows on the display. Note that, xe2x80x9csingle sample per pixelxe2x80x9d, as used herein, does not only refer to rendering one sample per pixel but rather implies that a single sample value contributes to the final output pixel value.
An alternate sample mode, such as a xe2x80x9csingle sample per pixelxe2x80x9d mode may be desirable for backwards compatibility of the multi-sampled sample buffer with legacy APIs such as X-Windows. X-Windows servers assume a traditional, single-sampled frame buffer when rendering. They provide the graphics system with pixel write commands assuming a single value per pixel. The graphics system of the present invention is capable of operating using the default multi-sampling mode for the whole display except for windows where a legacy API system, such as X-Windows, is outputting graphical data. For those windows, the graphics system of the present invention is capable of using a xe2x80x9csingle sample per pixelxe2x80x9d sample mode to ensure backwards compatibility. As used herein, the term xe2x80x9cmulti-samplingxe2x80x9d refers to effectively using more than a single sample to compute the value of a pixel, and the term xe2x80x9csingle sample per pixelxe2x80x9d refers to effectively using a single sample to compute the value of a pixel.
An alternate sample mode, such as the xe2x80x9csingle sample per pixelxe2x80x9d may also be desirable where the anti-aliasing effects of the default multi-sampling mode may not be desired. The graphics system of the present invention is capable of operating using the default multi-sampling mode for the whole display except for windows that are designated as xe2x80x9csingle sample per pixelxe2x80x9d windows. In those windows, the graphics system is capable of using a xe2x80x9csingle sample per pixelxe2x80x9d sample mode to maintain the xe2x80x9caliasedxe2x80x9d look of the graphics in that window. The anti-aliasing effects of multi-sampling may not be desired, for example, where additional sharpness and contrast are expected in the graphics data.
The graphics system may use various embodiments to implement the single sample per pixel mode. In one embodiment, for each pixel that has been designated to use a xe2x80x9csingle sample per pixelxe2x80x9d sample mode, the graphics processor may receive a single value corresponding to each pixel. This single sample value per pixel may be rendered into the sample buffer for pixels in a respective window or may be received from a legacy API such as X-Windows. In one embodiment, the graphics processor replicates the received value for all the samples within a bin corresponding to a pixel. Thus, the number of samples per pixel is the same within windows designated to use xe2x80x9csingle sample per pixelxe2x80x9d sample mode as with windows designated to use the default multi-sampling sample mode.
The sample-to-pixel calculation unit may or may not perform regular filtering to generate pixel values for xe2x80x9csingle sample per pixelxe2x80x9d windows. In one embodiment, the sample-to-pixel calculation unit selects the sample that is closest to the pixel center and assigns that value to the pixel. In another embodiment, the sample-to-pixel calculation unit performs regular filtering to generate the value for each pixel. Since the single sample value has been replicated for all sample values corresponding to the pixel, regular filtering essentially still only uses the single received sample value in generating the output pixel. Any filter type may be used, and preferably, the range of the filter is restricted within the area of the pixel or bin, i.e., no overlap of the filters occurs.
In another embodiment, for each sample, pixel, or bin that has been designated to use a xe2x80x9csingle sample per pixelxe2x80x9d sample mode, the graphics processor receives a single value corresponding to each pixel and only renders one sample per pixel. The rendered sample is assigned the received single value and may be rendered at or near the pixel center location. Thus, this method may use different or variable sample densities on a per window basis. During generation of the output pixels, the sample-to-pixel calculation unit reads the value of the single sample and assigns that value to the corresponding pixel without any filtering. Filtering is not necessary since there is only one sample corresponding to each pixel.
In yet another embodiment, the graphics processor uses the default super-sampling sample mode for all rendered samples, regardless of the sample mode. In other words, the graphics processor renders a plurality of different samples for each pixel regardless of the sample mode. For those samples corresponding to windows designated as xe2x80x9csingle sample per pixelxe2x80x9d sample mode windows, the sample-to-pixel calculation unit applies a filter that is restricted in area so as to only consider one sample for calculating the value of each pixel. Alternatively, the sample-to-pixel calculation unit may simply select the sample closest to the pixel center for those pixels. For other windows on the screen, the regular default filter may be applied when calculating the values of the pixels, e.g., a plurality of samples in the neighborhood of the pixel center are selected, and the filter is applied to the selected samples to generate the output pixel values.